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Abstract 

The use of the Internet to learn involves complex cognitive activities. Educational researchers claim more attention in 
studying the nature of students’ challenges when using digital information for learning purposes. Our research 
investigated in depth the challenges that secondary students face when solving web information-problem tasks. We 
thoroughly analysed how forty secondary students (18 girls and 22 boys) solved a scholar task using web information. 
The students’ actions on the computer screen were logged. Findings showed specific difficulties in solving digital tasks 
due to when (1) typing appropriate search terms, and when (2) selecting reliable and useful results from a “search 
engine results page”, which hinders the correct finding and understanding of web information needed to solve the 
digital task. In addition, the participants had a lack of reflection regarding the different actions to solve the digital task. 
Furthermore, our findings revealed that gender was not associated with differences in digital attainment. 

Keywords: Information-problem solving (IPS), secondary education, classrooms; web search, skills, performance, 
gender differences 

1. Introduction 

Students in Secondary Education use Internet for both leisure and school assignments. However, it is widely accepted 
that the complex cognitive activities involved in gathering and processing information from the Web are not 
instinctively acquired (Wood, 2009). Previous research has shown that students have difficulties in mastering these 
skills (Badia & Becerril, 2015; Bannert & Reimann, 2011; Walraven, Brand-Gruwel, & Boshuizen, 2008; Wood, 2009) 
and it is claimed that more research in studying the nature of these challenges is needed. This new research should fuel 
on well-designed and well-embedded scaffolds given to students in order to promote better students’ digital skills 
development which will allow them to reach a stronger educational attainment (e.g., Raes, Schellens, De Wever, & 
Vanderhoven, 2012). Our study falls in this line of research and the main goals are (1) to give insight about the 
challenges that secondary students face when solving tasks using web information to learn curricular contents at school, 
and (2) to seize gender differences on digital skills and task performance. These results will shed some light on 
instructional design to promote effective students’ digital skills development in classroom settings. 

To accomplish these two goals, we have divided this section in three main parts. First, we discuss the concept of digital 
literacy as an information-problem solving process. Second, we analyse previous studies on the students’ challenges in 
developing effective skills in solving information problems. Third, we review current literature about gender differences 
on the use of Internet in scholar learning. After the introduction section, we outline our research methodology. Finally, 
we analyse and discuss the results obtained in our study and its educational implications focusing on better-designed 
and better-embedded scaffolds. 

1.1 Digital Literacy as an Information-problem Solving Process 

Information-problem solving (IPS) has been defined as the process that involves complex cognitive activities to solve a 
problem using web information (Eisenberg & Berkowitz, 1990; Brand-Gruwel, Wopereis, & Vermetten, 2005; 
Brand-Gruwel, Wopereis, & Walraven, 2009; Brand-Gruwel & Stadler, 2011). 

The IPS process can be segmented into several skills (Brand-Gruwel et al., 2005): (1) An information problem is 
defined. (2) A search engine (i.e., Google) is usually selected, where a query with search terms is entered. Consequently, 
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a “search engine result page” (SERP) with a list of search result links is displayed to the user. (3) Those search results 
are assessed by checking the information that each result contains: i,e., title, summary, and “uniform resource locators” 
(URL). (4) After accessing a selected web page, its information is scanned, evaluated, and extracted for further 
processing in case of relevant information. (5) The information from different web pages is integrated in order to solve 
the problem. Each skill is decomposed into several skills. For instance, the skill “search for information” is constituted 
by the skills “specify search terms” and “judge results from a SERP”, as was proposed by Brand-Gruwel et al. (2005) in 
their IPS model (Figure 1). 



Figure 1. Information-problem solving (Brand-Gruwel et al., 2005; 2009) 

Several studies have reported that users of all ages, and especially secondary students, struggle in the development of 
IPS skills. In the next section, we review those challenges, and for this revision we will follow the skill decomposition 
model proposed. 

1.2 Challenges Faced by Secondary Students in Information-problem Solving Tasks 

In this section, we review the challenges that secondary students usually face while solving information problems. We 
divide this revision in two main parts: challenges related to the IPS process (constituent skills and skills), and 
difficulties to attain a final product. 

1.2.1 Information-problem Solving Process 

In reference to the constituent skill “Defining the problem’, previous research shows that secondary students have 
difficulties in the next four subskills: formulating questions (Wallace, Kupperman, Krajcik, & Soloway, 2000), 
activating prior knowledge, clarifying task requirements, and determining needed information (Walraven et al., 2008). 
Besides, most students start searching with no much reflection about the topic and the task and without outlining the 
search (Fidel et al., 1999, Ladbrook & Probert, 2011). 

In relation to the constituent skill “Searching for information”, secondary students have problems in selecting the 
appropiate “search terms” (Badilla-Quintana, Cortada-Pujol, Riera-Romam, 2011; MaKinster, Beghetto, & Plucker, 
2002; Van Deursen, Gorzig, Van Delzen, Perik, & Stegeman 2014; Wallace et al., 2000). Most of the times, students do 
a general search when they navigate the hypermedia environment rather than having specific searching objectives 
(Azevedo, Cromley, & Seibert, 2004). Furthermore, students do not systematically “judge the search results” and 
therefore they have difficulties to consult reliable web sources which have been chosen attending to superficial cues as 
the title or the web site briefing (Rouet, Ros, Goumi, Macedo-Rouet, & Dinet, 2011). 
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To “Scanning information”, teenagers also have problems in choosing between reputable and questionable sources, and 
selecting and assessing web site information (Lorenzen, 2002; Ladbrook & Probert, 2011; Van Deursen et al., 2014). As 
a result, the information selected may either come from a commercial or a scientific source (Fidel et ah, 1999; Mason, 
Ariasi, & Boldrin, 2011). 

Concerning the constituent skill “Processing information”, expert searchers spend more time on elaborating the content 
than novices (Brand-Gruwel et al., 2005). In the same line, Goldman (2011) found that the amount of time spent on a 
site was another key difference between novice and expert searchers. The former ones spent more time on reliable sites 
whereas the novice did not differentiate them. 

Finally, to “Organizing and presenting information”, students predominantly use ineffective strategies such as copying 
information from the hypermedia environment to their notes (Azevedo et al., 2004) and have the tendency to “copy and 
paste” the web information in order to solve the scholar tasks (Probert, 2009). In addition, Raes et al. (2012) also 
observed that students tend to reduce the whole task to find a straightforward answer on a particular web site instead of 
reading critically and thoroughly the web information. 

To sum up, there are experimental evidences that support secondary-school students face serious challenges during the 
IPS process which hinders potential learning outcomes by using Internet in classroom activities. Therefore, instructional 
design is needed to help students developing effective skills and sub skills to solve IPS task. 

1.2.2 Task Performance 

The learning outcome obtained in an IPS task can be presented as a writing essay or as a set of answers to specific 
questions (Jonassen & Kim, 2009), among others. In the literature, this type of learning outcome has also been named 
“task performance”. An essay requires content understanding and developing arguments during IPS task that can be 
used as an assessment instrument to value educational attainment. Content understanding is not only a recollection of 
facts and definitions associated with a particular subject area, but it can be viewed as a matter of degree in which an 
individual understands concepts, principles, structures, or processes at a relatively deep level and it is able to show 
certain behaviour patterns (Nickerson, 1995). Therefore, understandings could be made evident to others in terms of 
overt behaviours as learners communicate or act in five ways (Hakkarainen, 2003: 205-206): (1) Separated, Simple 
Low-Level Facts. (2) Partially-Organised Facts. (3) Well-Organised Facts. (4) Partial Explanation. (5) Explanation. 

Furthermore, another approach to assess students’ writing processes and learning attainment is the proposal made by 
made Scardamalia and Bereiter (1991), who made a distinction between knowledge telling and knowledge transforming. 
In a similar vein, Priemer and Ploog (2007), depending on the ways students used information obtained from the Web, 
they were classified into two types: compilers, who usually copied text from Web sources and lacked adequate 
processing of the information; and authors, who created an original and more elaborated text. 

To sum up, a written response to an information problem is a complex activity because writing using information that 
comes from multiple documents entails complex cognitive processes (Raes et al., 2012; Wiley & Voss, 1999), in which 
students of all ages face many difficulties that should be guided through a well-design educational intervention. 

1.3 Gender Differences 

Gender has been recognized as an important variable that influences skills related with computers (Chen & Macredite, 
2010; Young, 2000). Literature has suggested that females report higher levels of computer anxiety than their male 
counterparts; in addition, males achieve much better outcomes than females when using computers (Karavidas, Lim, & 
Katsikas, 2005). Chen and Fu (2009) analysed a sample of 1,409 secondary-school students in Northern Taiwan and 
pointed out that the frequency of Internet used by girls was significantly lower than by boys, and that boys and girls 
differed in their patterns of Internet use. Similarly, Tsai and Tsai (2010) reported that male students were more active 
Internet users than their female counterparts; however, boys used the Web mainly for exploration and girls had the 
tendency of using it for communication issues, as also reported by Jackson, Von Eye, Witt, Zhao, and Fitzgerald (2011). 

Large, Beheshti, and Rahman (2002), in a study with 11 year-old children, pointed out that boys were more actively 
engaged in navigating on Internet than girls. Particularly, boys submitted more search terms in search engines, explored 
the hyperlinks more frequently, visited more webpages per minute, gathered and saved information more often than 
girls. Consequently, the former visited less webpages as they spent more time viewing pages than boys (Arcand, Nantel, 
& Senecal, 2011) than their equals. 

In this line of research, Zhou (2014) analysed 107 Chinese university students’ perceptions about the use of Internet to 
learn and also found that males were more active in web searching than females, as they performed a larger number of 
searches and search queries. 

These findings are similar to those of Roy and Chi (2003), who showed that boys tended to execute more page shifts per 
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minute, by oscillating between entering searches and scanning the web pages excerpts returned (horizontal searchers), 
more than girls; conversely, girls had a tendency to actually open and browse entire linked document (vertical 
searchers). However, Lorigo, Pan, Hembrooke, Joachims, Granka, and Gay (2006) analysed gender differences through 
eye-tracking data and reported that boys’ navigation patterns were more linear than girls’, and these returned more often 
to previously visited abstracts when they searched web information. 

Furthermore, Tsai (2009) examined male and female’ search strategies in the behavioural, procedural and metacognitive 
domains through self-reported measures in a sample of 324 high school students. Gender difference was found in 
Internet control group. Therefore, although boys usually have higher self-assessed web skills, their current search skills 
may not be statistically different from girls (Hargittai & Shafer, 2006). 

Despite the differences between boys and girls patterns while working with digital information, these results are not 
conclusive as some other research studies did not show such differences. For example, in a study conducted with 340 
Greek high-school students were not find gender differences in Internet searching skills (Papastergiou & Solomonidou, 
2005). Likewise, Van Deursen and Van Diepen (2013) in a study with Dutch adolescents on Internet skills did not reveal 
any gender differences. In addition, Walhout Brand-Gruwel, Jarodzka, van Dijk, de Groot, and Kirschner (2015) neither 
found differences in task performance between girls and boys when learning in hypertext environment in an 
eye-tracking study. 

The above mentioned research on gender difference in navigation and Internet skills, not only has showed controversial 
results, but it has neither analysed the potential relation between web-search skills and task results. Consequently, our 
research pretends to fill this research gap by analysing the impact of the IPS skills used in task performance and gender 
variability. 

1.4 Research objectives 

The objectives of this study are the following three: 

(1) To analyse in detail the challenges that secondary students face while solving an information problem, regarding the 
IPS process - constituent skills and sub skills - and task performance. 

(2) To examine gender differences in IPS process and task performance. 

(3) To draw educational implications in better-designed instructions able to develop students’ efficient IPS by taking 
advantage of the Web potential as a resource for learning 

2. Method 

2. 1 Participants 

Forty students (18 girls and 22 boys) participated in this study. All of them were middle secondary aged 13/14 year-old. 
The students belonged to three urban schools in the city of Lleida (Spain) and they have not previously received any 
specific instruction in IPS skills and neither participated in any teaching innovation project involving the use of 
Information and Communication Technologies. 

2.2 Design and Procedure 

Students were evaluated by solving an IPS task. This task was designed by the teachers and the research group in order 
to assess students’ IPS process and task performance. The task was solved individually, in the students’ regular 
classroom and it lasted about 45 minutes. 

The IPS task was divided into two main parts. Part 1 was a fact-finding task in which participants were asked to 
complete a concept map and collect information about planet Mars (physical characteristics, orography, atmosphere, 
climate, satellites). Part 2 was carried out through an information-gathering task, in which participants had to 
accomplish three steps: (2.1) explaining the conditions a planet must have to make life possible by using information 
from the Web, (2.2) describing the favourable and unfavourable conditions of Mars, and (2.3) writing an argumentative 
essay about the possibility of installing a human colony on Mars and if so, what difficulties would have to be overcome 
and why. 

2.3 Data Instrument 

Students’ actions to solve the IPS task were logged in the real context of the classroom by using a screen capture 
software called CamStudio 2.0. 

2.4 Data Analysis and Procedure 

Regarding the anlysis of the IPS proces, we considered two types of information retrieved from the log files: constituent 
skills and sub skills. In reference to the task performance, we assessed the answers given to the different questions of 
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the task, and the final essay written by the participants. Table 1 shows a summary of the variables analysed and their 
measurement, and Figure 2 gives an overview of the inter relation between the variables and its iterative performance 
(Eisenberg & Berkowitz, 1990). 

Table 1. Variables analysed and their measurement 


Variables 

Measurement 

Process 

Constituent skills 

Time spent and frequencies 

1. Defining the problem 

2. Searching for information 

3. Scanning and processing information 

4. Organizing and presenting information 

Sub skills 

Appropriateness score 

5. Search terms 

6. Selected results 

Task performance 

7. Answers 

Correctness score 

8. Information-gathering answers 

Elaboration level 

9. Final essay 

Explanation level 


PROCESS TASK PERFORMANCE 



Figure 2. Information-problem solving model (Brand-Gruwel et al., 2005; 2009) 

One log file per student was obtained. Each log file was transcribed in detail and, as a result, a protocol was obtained 
from each log file. Each protocol consisted of three main parts. The first part was devoted to the constituent skills and 
contained the following columns: time line, constituent skill performed, and duration of the skill performed. The second 
part was devoted to the web search sub skills and consisted in the following columns: search terms typed, correct search 
terms, appropriateness of the search terms, selected results, correct selected results, and appropriateness of the selected 
results. The third part of each protocol was about the task performance and contained the following columns: students’ 
answers, answers correctness, elaboration level, and explanation level. 

2.4.1 Data Analyses of the Process Variables 

The first set of variables analysed referred to the constituent skills : (1) Defining the problem, (2) Searching for 
information, (3) Scanning and processing information, and (4) Organizing and presenting the information (Brand 
Gruwel et al., 2005; 2009). Each constituent skill was coded, including the time spent and the frequency. Short 
descriptions of these constituent skills are provided in Table 2. 


46 






























Journal of Education and Training Studies 


Vol. 5, No. 1; January 2017 


Table 2. Coding scheme of constituent skills 

Constituent skills Short descriptions 

Defining the problem The student analyses the demand in the IPS task (the computer screen shows 

the web page of the assignment demand), without typing anything. 

Searching for information The student searches for the information on the Web: accessing a search 

engine, typing search terms on it, or selecting results from a SERP (the 
computer screen shows the search engine or the SERP). 

Scanning and processing The student scans or processes the information on a web site, 
information 

Organizing and presenting The student types an answer on the web page of the assignment demand. 

information _ 

The second set of variables analysed referred to the following sub skills : search terms typed and selected results. Two 
reasons drove us to only consider these sub skills: first, several studies have reported on the relationship between the 
search terms appropriateness and the selected results’ evaluation with expertise in web searching (i.e., Lazonder, 2000), 
and also with the quality of the product (Willey, Goldman, Graesser, Sanchez, Ash, & Hemmerich, 2009) and second, 
the kind of data obtained (log files) allows observing specific actions made by the subject with the mouse or keyboard 
(for example: search terms typed, results selected by clicking on them, etc.). 

(1) Search Terms. Each search term typed was transcribed verbatim and the appropriateness of each search term was 
coded as “appropriate” (1 point) or “inappropriate” (0 points). For example, the search term “diameter of Mars” typed 
in Google when the participant was searching to answer what is Mars’ diameter, was considered “appropriate”, whereas 
“conditions of Mars for life” when the question was related to the conditions that make a planet suitable for life was 
coded as “inappropriate” because that search term referred only to Mars while the question’s aim made reference to any 
planet. In addition, an appropriateness score was calculated as a percentage, considering the number of search terms 
used and the number of appropriate search terms. 

(2) Selected Results. Each result that was selected from a SERP was gathered and also was coded as “appropriate” or 
“inappropriate”. The criteria taken into account to evaluate each search result were both usability and reliability 
(Authors, 2012; Gerjets, Kammerer, & Werner, 2011). A selected result was considered “usable” when its content (title, 
description, URL, and other information available in the SERP) followed the question to be answered, and was 
considered as “reliable” when the author or source was plausible. Each selected result was coded as “appropriate” (1 
point) when it was both usable and reliable, and as “inappropriate” (0 points) when it was either not usable or unreliable. 
Again, an appropriateness score was calculated as a percentage considering the number of total selected results and the 
number of appropriate ones. 

Two raters who were familiar with the search task, the materials and the coding scheme scored 20% of the protocols. 
Interrater reliability computed on this subsample of protocols yielded a Cohen’s kappa higher than .80. Finally, only one 
rater scored the remaining protocols. 

2.4.2 Data Analyses of the Task-performance Variables 

All the answers given and the final essay written by each participant were collected and evaluated. The evaluation of 
these answers was carried out in three different forms: (a) answer correctness, (b) elaboration level of the 
information-gathering answers, and (c) explanation level of the final essay. 

(a) Answer correctness. Each student’ answer was coded in a binary fashion as either “correct” (1 point) or “incorrect” 
(0 points). A correctness score was calculated by considering the percentage between the number of correct answers 
given and the maximum number of points to be obtained (16 points). 

(b) Elaboration level. 7 questions out of the 16 required an elaborated answer from the students. We wanted to 
characterize the different forms students used web information to respond these questions by coding students’ answers 
in eight grading categories, as summarized in Table 3. 
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Table 3. Coding scheme of elaboration level on the information-gathering answers 

Level of Category Short description: 

elaboration _ The student... _ 

1 No answer ... did not answer the question. 

2 Improperly Copy & Paste ...copied and pasted improper information to the question. 

3 Incorrect or incomplete ...gave an incorrect or incomplete answer to the question. 

4 Copy & Paste with ...copied and pasted a correct answer, but (s)he added 

Irrelevant Information irrelevant information that the question did not require. 


5 

Copy & Paste 



...copied and pasted a correct answer. 

6 

Elaboration on 

the content 

...answered the question by including one or more of the 
following elements: translation from another language, 
insertion of own words, enumeration of concepts. 

7 

Integration 

Information 

of 

the 

...answered the question by explaining with own words, 
comparing or linking concepts. 

8 

Integration 

Information 

of 

the 

and 

...answered the question by using some of the following 
elements: own argumentation, own comments, giving 


Argumentation examples, presence of connectors (because, although, it 

_ means, in such a way,...). _ 

(c) Final essay explanation level. This variable informs about the level of content understanding and reasoning present 
in students’ final essay. Each content idea of the final essay was classified using a four-step scale: (1) separate pieces of 
facts, (2) organized facts, (3) partial explanation, and (4) explanation. This scale was based on the scales proposed by 
Hakkarainen (2003). 

Each participant essay was reviewed by two raters. In those scores where differences were found, the final score was 
decided after a consensus. 

SPSS version 19.0 software was used for the data analysis, t-student was calculated, with confident intervals between 
95% and 99%, in order to identify differences considering the gender variable. 

3. Results and Discussion 

In this section, first we present and discuss the in-depth analysis of the challenges that secondary students face while 
solving an information-problem task, regarding the IPS process -constituent skills and sub skills- and task performance. 
Second, we study gender differences in the dependent variables of our study. 

3.1 In-depth Analysis of the Students ’IPS Challenges (Research Objective 1) 

Table 4. Constituent skills, sub skills and correctness of the task performance 


Variables 

M 

SD 

Constituent skills 



Time spent (in minutes) 



1. Defining the problem 

8.46 

2.53 

2. Searching for information 

8.30 

3.59 

3. Scanning and processing information 

14.23 

4.20 

4. Organizing and presenting information 

8.54 

5.18 

Total 

40.34 

7.15 

Frequencies 



1. Defining the problem 

31.8 

9.4 

2. Searching for information 

22.5 

10.6 

3. Scanning and processing information 

41.4 

16.1 

4. Organizing and presenting information 

16.5 

9.0 

Total 

112.2 

35.8 

Sub skills 



5. Search terms: Appropriateness score 

53.68 

21.62 

6. Selected results: Appropriateness score 

66.33 

21.08 

Task performance 



7. Answers: Correctness 

7.02 

4.40 


3.1.1 Constituent Skills 

As can be seen in Table 4 the entire activity lasted in average, 40.34 minutes (SD = 7.15), and was performed by 112.2 
constituent skills (SD = 35.8). Comparing the time spent and the frequency performing each skill, a high frequency of 
each skill performed can be appreciated. In other words, the participants clicked very fast to switch from one skill to 
another. This was particularly observed on the skills “Searching for information” and “Scanning and processing 
information”. Previous studies found that secondary students did have more iteration between “searching” and 
“scanning” than adults, and this behaviour was considered as an indicator of a lack of reflection (Brand-Gruwel et al., 
2009). 
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3.1.2 Sub Skills 

The average of the appropriateness score reached in the sub skills “search terms” typed in a search engine, and “selected 
results” from the SERP retrieved by a search engine, were 53.68 and 66.33, respectively. 

As in previous research, our participants got a low score in performing these sub skills appropiately (Walraven et al., 
2008). In this line, MaKinster et al. (2002) pointed out that formulating effective searching queries is difficult for most 
web users. Regarding selected results from a SERP, secondary students tend to click fast on a link without much 
reflection (Brand-Gruwel et al., 2009), and most of them have difficulties when evaluating web information in terms of 
relevance and reliability (Gerjets & Hellenthal-Schorr, 2008). In addition, it has been reported that novices are less 
efficient at locating useful web sites (Lazonder, 2000; MaKinster et al., 2002). 

It can be concluded that the participants analysed in this study presented patterns of novice information-problem solvers, 
characterized by a lack of reflection when shifting skills, typing search terms, and selecting results. 

3.1.3 Task Performance 

The average of students’ grade of correctness was 6.90 (SD = 4.29) which means that only 43.12% of the students’ 
answers were correct. 

In Figure 3, the percentage obtained in the elaboration level is displayed. As it can be observed, most of the students did 
not answer the questions (53.21%) and a high percentage (16%) answered incorrectly. 25% of the students’ answers 
were categorized as “elaboration on the content” and an insignificant percentage scored as “integration of the 
information” (Wiley, Goldman, Graesser, Sanchez, Ash, & Hemmerich, 2009). Contrarily to previous studies (i.e., Raes 
et al., 2012), the participants of our study used the copy-and-paste strategy in a low proportion (3.9%). This might be 
due to the specific requirements of the IPS task to be solved which demand an insightful elaboration of the answers. 


Integration of the Info 

0 , 0 % 


Elaboration on the 
content 
25,0% 


Integration-info & 
Argumentation 
0,4% 




Improperly Copy & Paste 
0,7% 


Figure 3. Task performance elaboration level 

Concerning the final essay explanation level written at the end of the IPS task, twenty-seven students (67%) did not 
answer. In addition, 33% of participants wrote their essay based on facts (categories 1, “separate pieces of facts”, and 2, 
“organised facts”), and none of the students were categorized as explanation fashion (categories 3, “partial explanation” 
and 4, “explanation”). These results show a low rate on content understanding (Hakkarainen, 2003) (Figure 4). 
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Figure 4. Final essay explanation level 

As a conclusion of this section could be stated that the participants of our study went through high challenges in task 
performance, since they reached low scores in correctness grade, elaboration level, and explanation level. 

3.2 Gender Comparison (Research Objective 2) 

As can be seen in Table 5, no remarkable statistical differences were found in gender in relation to constituent skills, sub 
skills, and task performance. 

Table 5. Constituent skills, sub skills, and task performance comparing girls and boys 


Variables 

Girls 


Boys 



Constituent skills 

M 

SD 

M 

SD 

P 

Time spent (in minutes) 

1. Defining the problem 

8.13 

3.19 

8.92 

2.62 

n.s. 

2. Searching for information 

7.32 

3.62 

9.13 

4.19 

n.s. 

3. Scanning and processing information 

15.39 

3.96 

13.23 

4.48 

n.s. 

4. Organizing and presenting information 

7.89 

5.44 

9.38 

5.21 

n.s. 

Total 

39.88 

7.97 

42.18 

7.10 

n.s. 

Frequencies 

1. Defining the problem 

31.33 

10.83 

32.18 

8.39 

n.s. 

2. Searching for information 

20.56 

8.68 

24.05 

11.84 

n.s. 

3. Scanning and processing information 

45.17 

14.52 

38.36 

17.07 

n.s. 

4. Organizing and presenting information 

16.78 

11.30 

16.32 

6.75 

.049 

Total 

114.56 

35.47 

112.05 

37.04 

n.s. 

Sub skills 

5. Search terms: Appropriateness score 

52.15 

24.19 

54.94 

19.75 

n.s. 

6. Selected results: Appropriateness score 

67.59 

21.83 

65.30 

20.91 

n.s. 

Task performance 

7. Answers: Correctness 

7.22 

4.44 

6.86 

4.46 

n.s. 


However, if we consider the elaboration level of the answers given by students, we can found some statistical 
differences (Figure 5) in the use of the “Copy & paste” strategy. Although the average was low, girls copied and pasted 
the information more frequently than boys (girls: M = .39, SD = .61; boys: M = .18, SD = .39; p = .014). In addition, 
girls answered some questions copying and pasting irrelevant information (girls: M = .06, SD = .24; boys: M = .00, SD 
= .00; p = .024) and some questions copying and pasting information in an improper way (girls: M = .11, SD = .32; 
boys: M = .00, SD = .00; p = .001). Contrarily to the findings of previous research (i.e., Chang, Chen, Huang, & Chou, 
2015), in our study girls used the “Copy & Paste” strategies more than boys. 
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Figure 5. Elaboration level on the information-gathering answers (*p < .05; **p< .01) 

In reference to the level of explanation of the final essay, any significant statistical differences were found between both 
groups, according the gender variable (girls: M = .33, SD = .485; boys: M = .32, SD = .568; p = n.s.). 

4. Conclusions and Educational Implications 

Our results revealed that secondary students face several challenges in the development of an efficient IPS process and 
task performance. In regard to variable gender, no important differences were found in our analyses. Both girls and boys 
have similar difficulties in solving information problems. Consequently, it is required to help students to develop better 
IPS skills and sub skills, and task performance. From our perspective, we consider that it is necessary to be aware of the 
difficulties that students usually face in IPS tasks in their classrooms, in order to support students to overcome such 
challenges. 

Considering the contributions made by our study regarding the secondary students’ challenges on IPS and the 
theoretical and methodological contributions of other studies, we propose that the design of an instructional process 
should be based on the following three instructional principles: (1) embedding the IPS skills in the scholar curriculum to 
leam the current subjects through meaningful and authentic contents, (2) structuring the students’ problem solving, and 
(3) supporting the students’ problem resolution by a series of scaffolds to guide the students’ progress. 

Related to the first instructional principle, IPS skills should be acquired and embedded within a relevant and meaningful 
learning content. Some effective embedded instruction research studies have showed a positive impact on learning IPS 
skills (Britt & Aglinskas, 2002; Raes et al., 2012), on enhancing students’ performance on domain contents (Raes et al., 
2012), and on transferring IPS skills to other contexts (Walraven, Brand-Gruwel, Boshuizen, 2010). 

As regards to the second instructional principle, structuring the instruction, it is worth highlighting the importance of 
guiding students through the whole process of information problem resolution. Scaffolding helps to accomplish all the 
steps needed to successfully solve a problem through providing a layer of structure between student and Internet (Segers 
& Verhoeven, 2009). A good structured instruction on IPS could be an instruction that follows the Inquiry-Web 
Learning methodology as proposed in WebQuest (Dodge, 1995). 

Related to the third instructional principle, supporting the instruction, we claim the necessity to design lessons which 
could provide scaffolds in each IPS skill and sub skill in order to tackle information problems. Sociocultural theory 
introduced the notion of scaffolding (Vygotsky, 1978; Wood, Bruner, & Ross, 1976) tailored to the needs of students 
with the intention of helping them to progress on their learning process. Educational research has already provided 
successful evidence about the improvement of IPS skills and individual learning in a technology environment by 
implementing appropriate questions and reflection prompts that trigger students to activate key cognitive processes to 
solve the learning task (Raes et al., 2012; Walhout et al., 2015). Scaffolds could be embedded in the IPS task to give 
guidance and support to leam IPS skills and sub skills, and gradually withdraw them as the instmctional process 
progresses. 

Following previous studies (Badia & Becerril, 2015; De Vries, Van der Meij & Lazonder, 2008; Morris et al., 2010; 
Pifarre, 2009; Pifarre, Cobos, & Argelagos, 2014; Stadtler & Bromme, 2008; Walraven et al., 2010, Wopereis, 
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Brand-Gruwel, & Vermetten, 2008), the instructional process might be scaffolded by means of questions, worksheets, 
scripts, and prompts (Stadler & Brome, 2008). For instance, scaffolds could be designed as it follows: (a) driving 
questions in order to encourage the activation of students’ prior knowledge; (b) prompts to perform a specific IPS skill, 
i.e., “write down the specific information you will need to solve this problem” (skill “defining the Problem”); (c) 
pop-up messages to guide a search; (d) worksheets to promote reflection about search terms or selection of the results 
(skill “searching for information”), and (e) maps to organize the information and writing templates (skill “organizing 
and presenting information”). 

The present study is an attempt to give an insight into the challenges of secondary students in the field of IPS and to 
draw educational guidelines for the design of IPS instruction in school. Nonetheless, our findings must be carefully 
interpreted due to the following two limitations. First, the technique used to collect the data was the log files technique, 
which may bring the risk to over interpret the actions made by the participants (Argelagos, Jarodzka, & Pifarre, 2011). 
As this technique only captures the actions made on the screen, some actions can easily be misinterpreted. The election 
of this technique was due to the ecological validity and the unobtrusiveness of the technique (Wopereis & van 
Merrienboer, 2011); however, the cognitive processes involved in IPS could be more in-depth analysed through other 
techniques as eye-tracking (Holmqvist, Nystrom, Andersson, Dewhurst, Jarodzka, & van de Weijer, 2011) or 
cued-retrospective reports (van Gog, Paas, van Merrienboer, & Witte, 2005) though they can only be applied in 
laboratory settings. Second, the sample considered for this study -40 students- might be a slightly limited segment. 

Notwithstanding the above limitations, this paper examined the challenges that secondary students face when solving a 
problem using digital information to learn curricular contents. Our findings draw inferences to help secondary students 
to enhance their IPS skills in everyday classrooms. We claim that this study might contribute to coaching secondary 
students into digital competences that may allow them an adequate use of web information. 
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